-
Rapid Response to Executive Order by HSR&D Investigators Leads to Widespread Adoption of Suicide Prevention Strategies
As part of the response to Presidential Executive Order #13861, this study analyzed a large body of text responses collected as a federal Request for Information (RFI) to uncover gaps in policy and practice across the U.S. in order to more effectively address and prevent suicide among Veterans. Findings identified unique barriers to care, in addition to ways in which data and research could be better aligned to improve outcomes. For example, Veterans need a greater connection to the community when they transition from military service and they want healthcare professionals who understand and/or have experience with military culture. Metrics for evidence-based risk prevention and treatment methods need to incorporate Veterans’ perspectives to ensure suicide prevention efforts address their needs and concerns. VA used the findings and policy opportunities identified from this RFI analysis to contribute to the development of The PREVENTS Roadmap with recommendations and implementation steps for suicide prevention. Thus far, the governors of 42 states and territories have submitted state proclamations to support the PREVENTS campaign and follow strategies outlined within the report. The HSR&D “QUICk: A Qualitative Interdisciplinary Collaboration” method developed for this study has been submitted for VA invention disclosure and could be useful for other researchers to address similar public requests for information or to help analyze other types of qualitative data to fulfill research mandates.
Date: March 3, 2022
-
Healthcare Supplement Features HSR&D Articles on the Importance of Embedded Research
This supplemental issue of the journal Healthcare: The Journal of Delivery Science and Innovation considers the impact of embedded research on solving the disconnect between healthcare organizations and health services researchers. In November 2019, a national meeting was organized by a trio of Federal funders – VA, the Agency for Healthcare Research and Quality (AHRQ), and the Patient-Centered Outcomes Research Institute (PCORI) – in collaboration with AcademyHealth and Kaiser Permanente to address the following themes: Organizational arrangements, including governance, staffing, and funding; Research support for management decisions; Data resources and use; Strengthening the embedded research community; and Accelerating implementation of embedded research output. Articles in this Supplement reflect the work that began during this national meeting.
Date: June 1, 2021
-
Supplement Features HSR&D Research on Combating Antibiotic Resistance – A Growing Worldwide Crisis
The research included in this collection links the methodological strengths of data science and transmission modeling, harnessing the unique advantages of VA’s large datasets to estimate key parameters governing colonization and transmission of pathogens from person to person. This Supplement demonstrates how the union of data science and transmission modeling can produce work that addresses some of the more difficult questions regarding the transmission and spread of antimicrobial resistance in different healthcare settings, leading to insights that can provide clinicians and policymakers with useful inputs in decision-making.
Date: January 15, 2021
-
New Tool Using Electronic Health Records Can Reliably Detect Infections after Cardiac Device Procedures
This study sought to develop and validate an electronic detection tool that accurately and reliably flags cardiovascular implantable electronic device (CIED) cases with true post-procedure cardiac device infection, leveraging the strengths of VA’s electronic health record. Findings showed that combining structured data, such as microbiology results, with text note searches was highly efficient for identifying true post-procedural infection. Among all 19,212 cardiac device procedures performed within VA in FY16-17, investigators reviewed 744 cases and identified 154 true procedure-related infections. The positive predictive validity of the tool was 44%, and overall sensitivity and specificity were 94% and 49%, respectively, indicating that the tool is useful for flagging cardiac device infection cases. This novel measurement tool, which adds data collected in clinical notes to flag cardiac device infections, has the potential to significantly reduce the burden of manual review for infection surveillance. Similar tools that combine structured data and key words from clinical notes could be developed to enhance infection detection, improve early event reporting, and support infection control efforts for other types of infections.
Date: September 21, 2020
-
Comparisons between VA and Non-VA Hospitals May Not Accurately Account for Mental Health Diagnoses
While CMS publishes performance metrics on Hospital Compare, the risk-adjustment algorithms underlying these metrics are often unclear. Further, recently published literature questions whether existing risk-adjustment algorithms accurately adjust for mental health comorbidities. This study sought to determine whether current risk-adjustment algorithms fairly compare VA hospitals with non-federal hospitals. In their analysis, investigators used the CMS Hierarchical Condition Categories (HCC) risk adjustment system version 21 (V21) because it is publicly available and has been used to adjust metrics published on the CMS Hospital Compare website. Investigators also measured mental health comorbidities using the PsyCMS (Psychiatric Case Mix System). Findings showed that current comparisons between VA and non-VA hospitals are flawed because the risk adjustment algorithms used to make patients comparable do not adequately control for mental health issues. Of 5,472,629 VA patients, the V21 model identified 694,706 as having mental health or substance use HCCs. The PsyCMS identified another 1,266,938 Veterans with mental health diagnoses. The top 10 mental health diagnoses missed by the V21 model included nicotine dependence (40%), depression not otherwise specified (31%), PTSD (27%), and anxiety (10%). Overall, the V21 model under-estimated costs for patients with low costs and over-estimated costs for patients with above average costs except for the top decile. For Veterans with a mental health diagnosis, the V21 model under-estimated the cost of care by $2,314 per patient. Risk scores may need to be developed based on a broader set of hospital data. Without such efforts, safety net hospitals, such as VA, may be penalized and patients and policymakers misled.
Date: December 14, 2018
-
Special Issue of Health Services Research Focuses on Linking VA and Non-VA Datasets to Improve Healthcare
The theme of this special issue of Health Services Research is linking VA and non-VA datasets. The articles in this special issue illustrate that researchers are increasingly linking diverse datasets as a valuable method for obtaining outcomes, treatments, and covariates that would otherwise be unavailable. All articles relied, in part, on the VA national database known as the Corporate Data Warehouse, which gathers data from VA’s electronic health record system; several articles also relied on VA-Medicare linked data provided by the VA Information Resource Center (VIReC) under a longstanding interagency agreement with the Centers for Medicare & Medicaid Services.
Date: December 1, 2018
-
Application of Triggers on VA “Big Data” may Help Identify Patients Experiencing Delays in Diagnostic Evaluation of Chest Imaging
Triggers offer one method to use big electronic health record (EHR) data to prevent and mitigate the impact of delays in care related to missed test results. Triggers consist of computerized algorithms that can scan thousands of patient records to flag those with clues suggestive of patient safety events. This study tested the application of a trigger within VA’s EHR to help identify delays in patient follow-up related to abnormal chest imaging results. Findings showed that the trigger identified delays in patient follow-up with a reasonable accuracy for use in the clinical setting, suggesting that triggers are able to identify almost all delays related to abnormal lung imaging follow-up, and cost-effectively minimize the amount of effort providers spend reviewing false-positive results.
Date: September 1, 2016
-
Data from Electronic Health Records Can Predict and Possibly Prevent Missed Patient Appointments
This study sought to develop a model that identifies patients at high risk for missing scheduled appointments (no-shows and cancellations), and to project the impact of predictive over-booking in a gastrointestinal (GI) endoscopy clinic – a resource-intensive environment with a high no-show rate. Findings showed that information from electronic health records can accurately predict whether patients will no-show. The model used in this study was able to correctly classify 711 out of 888 attended appointments, and 317 out of 538 missed appointments. The strongest predictor of no-show was a patient’s cancellation history – the proportion of all outpatient appointments missed. Veterans with histories of mood or substance use disorder, and those with a greater overall disease burden also were less likely to keep appointments. Predictors of being more likely to keep appointments included: being married, having a history of diverticular disease, attending a colonoscopy education class, and having care partly funded by VA. Urgency of appointment, race, ethnicity, and day of the week of appointment were not significant predictors of appointment no-shows. Compared to a strategy that employs a fixed level of overbooking, predictive over-booking was much less likely to lead to days where the clinic was substantially over- or under-booked.
Date: December 1, 2015
-
Majority of Male VA Enrollees Similar to Medicare Beneficiaries, Suggesting Greater Generalizability of Research Findings
This is the first study to assess the extent to which Veterans enrolled in VA healthcare have similar demographics and health characteristics as individuals with Medicaid, Medicare, and/or private insurance coverage. Findings showed that a majority of male Veterans enrolled in VA healthcare were similar in both demographics and health characteristics compared to Medicare beneficiaries, but this overlap decreased when comparing Veterans to individuals enrolled in Medicaid or those with private insurance. The proportion of overlap was 21%, 34%, and 41% for the Medicaid, privately insured, and Medicare comparison populations, respectively. When restricting the analysis to males, the proportion of overlap increased to 28% for Medicaid enrollees, to 39% for privately insured adults, and to 55% for Medicare beneficiaries. When restricting the analysis to elderly males (age 65+), the proportion of overlap increased from 55% to 65% for Medicare beneficiaries, suggesting that 2 of every 3 elderly male VA enrollees had a male beneficiary enrolled in Medicare sharing a similar set of characteristics. Findings of intervention studies conducted among VA healthcare enrollees may be generalized to some non-VA populations, particularly male Medicare enrollees. Further, effective interventions developed in elderly and/or male non-Veteran populations may be applicable to comparable VA users.
Date: November 20, 2015
-
Study Compares Data Sources for Provider Financial Incentives
This study examined how well data from automated processing of EHRs (AP-EHR) reflect data collected via manual chart review, and assessed the potential impact of data collection methods on incentive earnings for physicians and provider groups participating in a trial evaluating pay-for-performance for hypertension care. Findings showed that the total amount of incentives disbursed to providers would have been lower (by 10%) using the AP-EHR data to reward performance because this method under-reported the number of Veterans receiving appropriate medications – compared to manual review. Regarding how well the AP-EHR reflect data from manual review, results show almost perfect agreement for the BP control measure: manual review indicated 70% of Veterans had controlled BP compared to 67% by AP-EHR review. Moderate agreement was found between the data sources for the use of guideline-recommended anti-hypertensive medication: manual review showed 72% of Veterans were considered to have received guideline-recommended anti-hypertensive medications compared to 65% by AP-EHR. And low agreement was found for the appropriate response to uncontrolled BP: manual review showed that 52% of Veterans received an appropriate response for uncontrolled BP compared to 40% by AP-EHR review. Given the large amount of resources needed for chart review endeavors, investigators feel that a 10% difference in the total amount of incentive earnings disbursed based on AP-EHR data compared to manual review is acceptable.
Date: October 1, 2015
-
NEJM Perspective Discusses Withholding of CMS Data Related to Substance Use Disorder and Its Impact on Research
In November 2013, the Centers for Medicare and Medicaid Services (CMS) began to withhold from research data sets any Medicare or Medicaid claim with a substance use disorder (SUD) diagnosis or related procedure code. This move — the result of privacy-protection regulations overseen by the Substance Abuse and Mental Health Services Administration — affects about 4.5% of inpatient Medicare claims [recent research suggests this figure is closer to 7%] and about 8% of inpatient Medicaid claims from key research files, impeding a wide range of research evaluating policies and practices intended to improve care for patients with substance use disorders. As a consequence, VA researchers cannot see the full utilization of Veterans who also use Medicare- or Medicaid-financed healthcare. This Perspective summarizes the problem, quantifies it, describes how it arose, and argues that research access to such data should be restored.
Date: April 15, 2015
-
Factors Affecting Patient Test Results Follow-Up within VA’s Electronic Health Record System
This study sought to identify facility-level contextual factors that increase or decrease the risk of missed test results in 40 VA healthcare facilities from across the U.S. Findings showed that primary care providers (PCPs) at VA facilities that used additional strategies or systems to prevent missed test results preceived less risk of missing test results. However, few VA facilities used monitoring strategies to prevent missed test results. For example, facilities monitored follow-up of certain test results only when they considered them “critical” (e.g., x-ray suggestive of malignancy), but the processes for doing so were highly variable. Qualitative analysis identified three high-risk scenarios for missed test results: 1) alerts on tests ordered by trainees (important because 78% of VA facilities were training sites for one or more medical residency programs); 2) alerts “handed off” to another covering clinician; and 3) alerts on patients not assigned in the electronic health record to a PCP. Interventions to reduce missed test results might need to target organizational factors and not just individual providers; for example, monitoring systems to track missed test results.
Date: November 11, 2014
-
VA’s “Big Data”: Benefits and Challenges
This paper provides an overview of VA’s evolving approach to “big data” and illustrates how advanced analytics support clinical activities, with particular emphasis on the Patient-Aligned Care Team (PACT) model of patient-centered primary care. It also shares some of the challenges, concerns, responses, and future plans that have emerged from these initiatives.
Date: July 9, 2014
-
Prediction Model Using VA Data May Help Identify Primary Care Patients at Increased Risk for Hospitalization or Death
In an attempt to identify high-risk patients, investigators in this study developed statistical models using health information from VA’s clinical and administrative databases to predict the risk of hospitalization or death among all Veterans who were assigned to a primary care provider as of 10/1/10. Findings showed that prediction models using electronic clinical data accurately identified Veterans receiving VA primary care who were at increased risk of hospitalization or death. Of the top 5% of Veterans in terms of predicted risk, 51% were hospitalized or died within the following year. Predictors of death were quite different from predictors of hospitalization. In general, clinical and demographic characteristics (i.e., increasing age, metastatic cancer) were most predictive of death, while recent use of health services was most predictive of hospitalization. The authors suggest that in clinical settings, these values can be used to identify high-risk patients who might benefit from care coordination and special management programs, such as intensive case management, telehealth, home care, specialized clinics, and palliative care.
Date: April 1, 2013
-
Natural Language Processing with Electronic Medical Record Improves Identification of VA Post-Operative Complications
This study evaluated a natural language processing (NLP) search approach to detect post-operative surgical complications within VA’s electronic medical record (EMR). Findings showed that, among Veterans undergoing inpatient VA surgery, NLP using the EMR greatly improved the identification of post-operative complications compared to an administrative-code based algorithm. NLP correctly identified 82% of acute renal failure cases compared with 38% for patient safety indicators; 59% vs. 46% for venous thromboembolism; 64% vs. 5% for pneumonia; 89% vs. 34% for sepsis; and 91% vs. 89% for post-operative MI. An accompanying Editorial states that NLP has the potential to greatly enhance the EMR with new applications, such as automated quality assessment to assist in the performance of comparative effectiveness research.
Date: August 24, 2011
-
Using Administrative Data to Measure Treatment for Veterans with PTSD May Overestimate Delivery of Psychotherapy
This study sought to determine whether using administrative data to determine the number of psychotherapy sessions Veterans receive is equivalent to manual record review. Manually-classified notes were used to develop an automated coding protocol using the Automated Retrieval Console (ARC), a VA-developed natural language processing program. ARC was then used to independently code the notes, and the performance of the automated coding program was compared to manual coding. Findings showed that, of the notes that were administratively coded as individual psychotherapy for PTSD, 57% were coded as individual psychotherapy after manual review of records. Thus, nearly half of the encounters that would have been counted as the provision of psychotherapy in large administrative studies appeared to be records of services other than psychotherapy (e.g., intakes, psychological testing). Findings suggest that using counts of administrative codes over-estimates the amount of psychotherapy delivered to Veterans with PTSD. This suggests a potential limitation in current studies of the quality of care for PTSD in VA. The ARC program replicated the performance of the manual coders in classifying psychotherapy notes very well. This suggests that ARC may help bridge the gap between the accuracy of manual coding and the scope of administrative coding.
Date: February 14, 2011
-
Affective Disorders Strongest Predictor of Suicidal Behavior in Elderly Veterans Receiving Anti-Epileptic Medication
In January 2008, the FDA issued an alert indicating that anti-epileptic drug (AED) treatment is associated with increased risk for suicidal ideation, attempt, and completion. This study sought to assess variation in suicide-related behaviors in a population not well-represented by the data used for the FDA analysis – individuals 66 years and older with new exposure to AEDs. Findings show that in older Veterans who were started on AED monotherapy, the strongest reliable predictor of suicide-related behaviors was the diagnosis of an affective disorder prior to AED treatment. Increased suicide-related behaviors were not associated with individual AEDs. However, while most Veterans in this study received AED prescriptions for gabapentin (76.8%), a trend for increased suicide-related behaviors was found among those prescribed levetiracetam or lamotrigine, but interpretation was difficult since few Veterans received either drug (0.6%). The associations between suicide-related behaviors and chronic pain or chronic disease burden were not statistically significant, but dementia was significantly associated with suicide-related behaviors (42.2% with dementia vs. 25.8% without).
Date: January 11, 2010
-
Validity of Mental Health Diagnosis Using VA Administrative Data
This study estimated the validity of eight ICD9-based algorithms for the identification of mental health disorders in administrative data among 124,716 Veterans with diabetes who used the VA healthcare system in 1998, and also participated in the 1999 Large Health Survey of Veteran Enrollees, which included questions about history of mental health diagnoses. Findings show that many Veterans with a diagnosed mental health disorder can be identified through VA administrative data; however, the choice of algorithm influenced conclusions. Since the limitations of administrative data cannot be fully eliminated with any algorithm, the authors suggest that investigators and quality improvement programs also consider conducting sensitivity analyses in which they vary the algorithm, in order to indicate how different assumptions affect conclusions.
Date: January 1, 2010
-
Federal Investment in Electronic Medical Records
The American Recovery and Reinvestment Act (ARRA) includes $19 billion in incentives for the adoption of electronic medical records (EMRs) and $50 billion to promote health information technology. Medicare physicians adopting and making “meaningful use” of EMRs in 2011 and 2012 will be eligible for an initial payment of up to $18,000, with reduced payments in 2013 and 2014. However, current EMR systems’ inability to learn from aggregated health data has led to implementations and hospital information technology departments that can actually obstruct quality improvement. For example, much of the information contained in EMRs is formatted as unstructured free text – useful for essential individual communication but unsuitable for detecting quantifiable trends. This commentary suggests that the Department of Health and Human Services capitalize on the opportunity to mandate EMRs that have the potential to learn from data in the EMR system.
Date: September 9, 2009
-
“Rights” of Safe Electronic Health Record Use
This JAMA Commentary proposes eight “Rights” of safe electronic health record (EHR) use, which are grounded in an engineering model that addresses work-system design for patient safety. The authors recommend the use of the eight “Rights,” in order to address the complex interaction of organizational, technical, and cognitive factors that affect the safety and effectiveness of EHRs.
Date: September 9, 2009
-
Focus Groups Recommend Strategies to Decrease Missed Test Results
This paper reports on the efforts of two focus groups that formed as part of the Diagnostic Error in Medicine – A National Conference, which was held by the American Medical Informatics Association in 2008. Clinicians who were part of the focus groups were asked to develop interventions that might decrease the risk of diagnostic delay due to missed test results in the future. The focus groups concluded that while the electronic medical record helps to improve access to test results, eliminating all missed test results would be difficult to achieve. However, they did recommend several strategies that might decrease the rates of missed test results, including: improving standardization of the steps involved in the flow of test result information, greater involvement of patients to insure the follow-up of test results, and systems re-engineering to improve the management and presentation of data. They also suggest that healthcare organizations focus initial quality improvement efforts on specific tests that have been identified as high-risk for adverse impact on patient outcomes, such as tests associated with a possible malignancy or acute coronary syndrome.
Date: September 1, 2009
-
Assessing Accuracy and Completeness of Research Data
VA benefits from one of the most highly developed health information systems in the world, which includes the Immunology Case Registry (ICR) that was designed to monitor costs and quality of HIV care, and the Decision Support System (DSS) that was developed to monitor utilization and costs of Veterans in care. This study compared ICR and DSS datasets, which share overlapping laboratory data from the same VA electronic record system. Findings show that six of the laboratory tests for HIV patients that were studied demonstrated remarkably similar amounts of overlap (68% to 72%) between the two datasets, showing that ICR and DSS are both good sources of data for these tests. However, several other tests demonstrated much lower proportions of overlap (between 20% and 31%). These findings indicate that validation of laboratory data should be conducted prior to its use in quality and efficiency projects.
Date: January 1, 2009
-
Using VA Medical Data Alone May Underestimate Post-Stroke Depression and Geographic Variation in this Condition
When VA medical data alone were used, investigators found no significant geographic variation in the detection of post-stroke depression (PSD). But when VA medical data were used along with Medicare and VA pharmacy data, significant geographic variation (nearly double – 39.1% vs. 20.0%) was observed. This suggests that to gain a comprehensive view of PSD detection in VA patients, investigators must evaluate non-VA data sources because 70% of VA stroke patients were multiple health program users.
Date: December 1, 2008
-
Association between Nurse Staffing Levels and Patient Mortality in VA Hospitals
RN staffing was not significantly associated with in-hospital mortality for veterans with an ICU stay; however, increased RN staffing was significantly associated with decreased mortality among non-ICU patients. Continuing to estimate the effect of RN staffing and skill mix on patient outcomes using hospital-level data will provide poor estimates of outcome associations, such as in-hospital mortality.
Date: September 1, 2008
|
